3 links tagged with all of: prompt injection + agent security
Click any tag below to further narrow down your results
Links
Gray Swan cofounders Zico Kolter and Matt Fredrikson explain why AI systems need a different security mindset, focusing on indirect prompt injection, agent vulnerabilities and correlated failures. They walk through automated red teaming tools like Shade and the Gray Swan Arena, discuss guardrails, and argue that bigger models aren’t inherently safer and require bespoke security, identity management, and compliance measures.
- Human red-teamers ranked only fourth in robustness testing against browser-based agents, behind specialized automated red-teaming models.
- Scaling model size doesn't automatically improve safety, and agents introduce new vulnerabilities distinct from traditional IT security risks.
- The "lethal trifecta" (untrusted data, private data, exfiltration paths) creates attack surfaces that make a major prompt-injection breach feel inevitable.
- Effective AI defense will require machine-driven interpreters and agent-native identity/permissions systems, since humans can't keep pace with automated attack tools.
Trail of Bits crafted four malicious agent skills that bypass Cisco, Vercel, and ClawHub scanners by padding code or embedding payloads in docx and bytecode. The article shows how Siri-AI’s integration with private cloud compute and search can expose messages, emails, and calendars through prompt injection. It also covers a PyPI worm that auto-executes a hades-setup.pth hook to steal cloud and GitHub tokens, urging audits and immediate token rotation.
- Trail of Bits bypassed Cisco, Vercel, and ClawHub agent-skill scanners using tricks like 10,000 newlines of padding, or hiding payloads in .docx files and compiled Python bytecode—none of the scanners caught the buried triggers.
- Siri-AI's use of Gemini via Apple Private Cloud Compute pulls in messages, email, notes, and calendars, making it vulnerable to prompt injection that can leak private data or trigger unwanted actions like forwarding sensitive material.
- Socket found a PyPI supply-chain attack hijacking 19 packages (37 malicious wheels) that drop a hades-setup.pth hook to run an embedded Bun runtime, stealing AWS/GCP/Azure, GitHub, npm, and SSH credentials and exfiltrating them to GitHub repos while disguising traffic as Anthropic API calls.
- Affected package names include bramin, executor-engine, executor-http, funcdesc, coolbox, dynamo-release, and magique—any match warrants full credential rotation and a CI/CD log audit.
SkillSpector scans AI agent skills for vulnerabilities, malicious patterns, and CVEs before installation. It combines fast static checks with optional LLM analysis, covers 64 patterns across 16 categories, and outputs risk scores and detailed reports.
- SkillSpector scans AI agent skills before installation, checking 64 vulnerability patterns across 16 categories (prompt injection, data exfiltration, privilege escalation, etc.) plus real-time CVE lookups via OSV.dev.
- It runs a fast static scan by default, with an optional deeper LLM-based semantic check using providers like OpenAI, Anthropic, or NVIDIA's build.nvidia.com.
- Scans output a 0-100 risk score with severity labels and suggested fixes, in formats including JSON, Markdown, or SARIF for CI/CD pipelines.
- Even without LLM analysis (--no-llm flag), it still catches known code patterns and CVEs.